Image pickup elements such as CCD sensors and CMOS sensors include various types of color filter. A typical example such a color filter includes a color filter having a combination of primary colors (red, green, and blue) or complementary colors (cyan, magenta, and yellow).
FIG. 13 is a diagram illustrating primary-color Bayer arrangement of an image pickup element. In a matrix including four pixels, a pixel corresponding to red (R) and a pixel corresponding to blue (B) are diagonally arranged and pixels corresponding to green (G1 and G2) are arranged in the remaining two pixels. This arrangement pattern is repeated.
When an object includes a high-frequency component which exceeds resolution capability of the image pickup element, an aliasing signal is generated in an image signal generated by the image pickup element due to influence of the high-frequency component. Therefore, various methods for suppressing generation of an aliasing signal have been proposed.
For example, in order to suppress generation of an aliasing signal, a method for generating a luminance signal only using signals corresponding to G (G1 and G2) pixels without using signals corresponding R and B pixels in the primary-color Bayer arrangement shown in FIG. 13 has been proposed.
First, among signals corresponding to R, G, and B pixels obtained by digitalizing signals output from the primary-color Bayer arrangement of the image pickup element, values of signals other than signals corresponding to the G pixels are determined to 0. Next, vertical low-pass filter (V-LPF) processing is performed to limit a frequency band in a vertical direction and horizontal low-pass filter (H-LPF) processing is performed to limit a frequency band in a horizontal direction. By this, signals which have been compensated for using the signals corresponding to the G pixels are generated in the pixels, and a luminance signal corresponding to the G pixels is obtained. Hereinafter, a signal obtained through this method is referred to as a “first luminance signal”.
Alternatively, values of signals other than signals corresponding to the R pixels may be determined to 0 and the V-LPF processing and the H-LPF processing may be similarly performed so as to generate a luminance signal corresponding to the R pixels. Furthermore, values of signals other than signals corresponding to the B pixels may be determined to 0 and the V-LPF processing and the H-LPF processing may be similarly performed so as to generate a luminance signal corresponding to the B pixels. Then, the R luminance signal and the B luminance signal may be added to the G luminance signal and a resultant signal may be determined as a first luminance signal.
Moreover, in order to suppress generation of an aliasing signal, a method for generating a luminance signal using all color signals included in the primary-color Bayer arrangement shown in FIG. 13 has been proposed.
The V-LPF processing which limits the frequency band in the vertical direction and the H-LPF processing which limits the frequency band in the horizontal direction are performed using the signals corresponding to the pixels of all colors i.e., the signals corresponding to the R, G, and B pixels obtained by digitalizing signals output from the image pickup element having the primary-color Bayer arrangement while colors of the signals are not distinguished from one another. In this way, a signal is newly obtained. Hereinafter, the signal obtained through this method will be referred to as a “second luminance signal”.
FIG. 14 is a diagram illustrating characteristics of a spatial frequency in which the first and second luminance signals can be resolved.
An x axis represents a frequency space in a horizontal (H) direction and a y axis represents a frequency space in a vertical (V) direction. The farther a position from the original is, the higher the spatial frequency is.
A limit of resolution of the first luminance signal generated only using the signals corresponding to the G pixels in the horizontal and vertical directions is equal to Nyquist frequency (π/2) of arrangement of the G pixels. However, since lines which do not include a G pixel exist in an oblique direction, a limit resolution frequency in the oblique direction is lower than those in the horizontal and vertical directions. Accordingly, a region 1501 of a diamond shape shown in FIG. 14 corresponds to a resolution-available spatial frequency.
This is true for a case where the first luminance signal is generated by synthesizing R, G, and B luminance signals since the G luminance signal generated only using the signals corresponding to the G pixels has the highest resolution among the R, G, and B luminance signals.
On the other hand, since the second luminance signal is generated using the signals corresponding to the pixels of all the colors, when an object is achromatic, a square region 1502 shown in FIG. 14 corresponds to a resolution-available spatial frequency. Unlike the first luminance signal, since all lines have color pixels also in the oblique direction, a resolution-available spatial frequency in the oblique direction is higher than that of the first luminance signal. However, when the object is read, for example, signals output from pixels other than the R pixels are negligible, and therefore, only resolution in a region 1503 is obtained which is a quarter of resolution obtained in the case of an achromatic object.
In order to suppress generation of an aliasing signal included in an image signal, a method for generating a luminance signal taking characteristic of such first and second luminance signals into consideration has been proposed.
For example, a method for generating a luminance signal by changing a mixing ratio of the first luminance signal to the second luminance signal depending on a determination as to whether an object is a black-and-white object or a colored object has been proposed (refer to Japanese Patent Laid-Open No. 2003-348609).
Furthermore, a method for generating a luminance signal by changing a mixing ratio of the first luminance signal to the second luminance signal depending on a determination as to whether an object has the high correlation in the oblique direction as shown in FIG. 14 has been proposed (refer to Japanese Patent Laid-Open No. 2008-072377).
However, although these methods have advantages in terms of suppression of generation of an aliasing signal, other noise signals are not suppressed.
For example, in recent years, pixels of image pickup elements have been miniaturized. Therefore, in some cases, noise is increased due to the miniaturized pixels. Although various methods for suppressing generation of the noise by performing signal processing have been proposed, a fact that the suppression of generation of noise causes blur of images is widely known.
To avoid this situation, a method for suppressing generation of noise by dividing an image signal into a plurality of frequency components has been known (refer to Japanese Patent Laid-Open No. 2008-015741). Furthermore, a method for generating size-reduced image signal and synthesizing the size-reduced image signal and an original signal with each other so as to suppress generation of noise has known (refer to Japanese Patent Laid-Open No. 2009-199104).
For example, size-reduction processing is performed on an input image signal so that a minified image including frequency components lower than those of an input image is generated. Then, an edge intensity is detected from a minified image signal having the low frequency components, and a region in which an edge component is to be held is obtained in accordance with the edge intensity. Thereafter, an original image signal and the minified image signal having the low frequency components are synthesized with each other while various weights are applied to regions so that an image included in the region in which the edge component is to be held is not blurred whereby another image signal is generated.
However, an aliasing signal is not taken into consideration in this method for suppressing generation of noise by synthesizing image signals having a plurality of frequency bands.
When a single board sensor such as a primary-color Bayer arrangement is used, each pixel has all color signals without performing compensation by downsampling processing at a time of band division.
The reason thereof is described below. Here, a case where the downsampling processing is performed on half of pixels in the horizontal direction will be described.
It is assumed that color filters R, G, R, G, and so on are arranged in this order in a certain row. In this row, when only R signals are focused on, the R signals which have been arranged in every other pixel are arranged in every pixel through the downsampling processing. Furthermore, also when only G signals are focused on, the G signals which have been arranged in every other pixel are arranged in every pixel through the downsampling processing. Note that, in the downsampling processing performed on the G signals, values are obtained by calculating average values using adjacent pixel values so that gravity positions of the G signals correspond to those of the R signals. As described above, by performing the downsampling processing, all the R, G, and B color signals are generated in the same pixel positions without compensation. Hereinafter, this process of generating a plurality of types of signal in the same pixel positions is referred to as “synchronization”.
Since image signals which have been subjected to the downsampling processing are subjected to the synchronization, an original image signal should be subjected to the synchronization in order to synthesize the original image signal with an image signal which has been subjected to the downsampling processing. However, irrespective of the downsampling processing performed to suppress generation of noise, when the original image signal is subjected to the synchronization, an aliasing signal included in the original image signal may be superposed on the synthesized image signal.
The present invention has been made in view of the above problem, and it is an object of the present invention to provide an image processing apparatus which performs noise processing by dividing an input signal into a plurality of signals in frequency bands and which is capable of suppressing generation of an aliasing signal included in an image signal which is generated by sampling performed by an image pickup element.